Most digital cameras and camcorders have a single image sensor, such as a charge coupled device (CCD) image sensor or a complementary metal-oxide semiconductor (CMOS) image sensor. These image sensors use a color filter array or mosaic, which is an assortment of different spectral filters, formed in front of the CCD or CMOS image sensor for color acquisition.
A commonly-used color filter array or mosaic is the Bayer mosaic shown in FIG. 1. As shown, the Bayer mosaic includes color filters of the three primary colors red (R), green (G), and blue (B), where the green (G) color filters are arranged in a checkerboard pattern and the red (R) and blue (B) color filters are arranged in line sequence. One reason tri-chromatic filter arrays are used is that tri-chromatic sensing is near-sufficient in terms of colorimetric color reproducibility. It is also commonly assumed that this pixel assortment is the only practical approach for sensing color information with a semiconductor image sensor. However, the Bayer mosaic is limited in its capacity because it provides a limited amount of spectral information. That is, the Bayer mosaic provides spectral information for the three colors red (R), green (G), and blue (B). In addition, while interpolation and other techniques are available to fill in missing spectral information, these approaches typically provide a resulting image showing color aliasing and other artifacts. For example, FIG. 2 shows the differences between a ground truth image 210 and an image 220, which suffers from color aliasing and other artifacts resulting from a bicubic interpolation applied to signals captured using the Bayer mosaic. Dashed region 222 identifies the portion of image 220 that is affected by color aliasing and other artifacts.
In recent years, new image sensing technologies have emerged that use pixel assortments to enhance image sensing capabilities. For high dynamic range (HDR) imaging, a mosaic of neutral density filters with difference transmittances has been used. This approach to high sensitivity imaging builds upon the standard Bayer mosaic by using panchromatic pixels that collect a significantly larger proportion of incident radiation.
Despite these advances, the previously described mosaics and camera systems have limitations. For example, these mosaics and camera systems are used to generate one specific type of output image.
Accordingly, it is desirable to provide generalized assorted pixel camera systems and methods that overcome these and other deficiencies of the prior art.